Skip to content

[https://nvbugs/6507114][fix] Extend the per-model qwen3.5_moe_35b.yaml with…#16868

Open
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6507114
Open

[https://nvbugs/6507114][fix] Extend the per-model qwen3.5_moe_35b.yaml with…#16868
trtllm-agent wants to merge 1 commit into
NVIDIA:mainfrom
tensorrt-cicd:repair-bot-bug6507114

Conversation

@trtllm-agent

@trtllm-agent trtllm-agent commented Jul 25, 2026 •
edited by coderabbitai Bot
Loading

Copy link
Copy Markdown
Collaborator

Summary

  • Root cause: Two independent AutoDeploy compile-pipeline defects stack on this test: multi_stream_moe/gemm stall the SM90 TP=4 GSM8K "Fetching responses" loop, and default piecewise CUDA-graph capture silently kills a worker on the first multi-image MMMU prefill for the Qwen3_5MoeForConditionalGeneration VLM.
  • Fix: Extend the per-model qwen3.5_moe_35b.yaml with compile_model.piecewise_enabled: false alongside the multi_stream_moe/gemm disables (scoped to this model config only, matching super_v3.yaml/super_v3_mtp.yaml precedents); remove the stale waives.txt entry.
  • Automated fix generated by repair-bot

Test plan

  • Verify fix on the same GPU type as the original failure
  • Check for regressions in related tests

Links

Dev Engineer Review

  • Disabled piecewise CUDA-graph capture and multi-stream MoE/GEMM transforms for Qwen3.5-35B, addressing known GSM8K stalls and MMMU prefill worker termination.
  • Changes are scoped to the model configuration, retain stage: compile, and align with model-specific precedents.
  • Removed the stale Qwen waiver entry from waives.txt; formatting and ordering remain consistent.
  • No public API or code changes; no apparent regressions.

QA Engineer Review

  • Test-list-only change: removed accuracy/test_llm_api_autodeploy.py::TestQwen3_5_397B_MoE::test_bf16_small[4] from tests/integration/test_lists/waives.txt.
  • No test-db/ or qa/ files were modified.
  • End-to-end validation passed on 4×H100: MMLU 84.75, GSM8K 88.97, and MMMU 59.89.
  • Verdict: needs follow-up because CBTS coverage data is unavailable.

@HandongLi-01
[nvbugs/6507114][fix] Disable piecewise CG and multi_stream on Qwen3.…
a90d01c 
…5-35B AutoDeploy

TestQwen3_5_397B_MoE::test_bf16_small[4] runs MMLU + GSM8K + MMMU on
Qwen3.5-35B-A3B via AutoDeploy on 4xH100. The default per-model config
(qwen3.5_moe_35b.yaml) opts into two performance transforms that both
break for this smoke test:

1) multi_stream_moe / multi_stream_gemm overlap shared-expert compute
   and all-reduce onto an aux stream; on SM90 TP=4 this stalls GSM8K
   "Fetching responses" at 0/1319 (the original reproduce failure).

2) compile_model.piecewise_enabled (inherited from default.yaml) hits
   an untested edge case in the VLM prefill path -- MMMU multi-image
   submits kill a worker silently at 84/900 with no Python traceback
   or CUDA error. Piecewise + VLM was previously hardened by NVIDIA#14006
   (commit 19b5d0e) but that fix did not cover this combination
   (Qwen3_5MoeForConditionalGeneration + mrope 3D position_ids +
   fewshot_as_multiturn chat template).

Disable both in the per-model YAML, matching the existing precedent in
model_registry/configs/super_v3_mtp.yaml (piecewise_enabled: false for
MTP incompatibility) and super_v3.yaml (multi_stream_moe/gemm: false).
The disables are scoped to this model config only and do not affect
qwen3.5_moe_400b.yaml or other AutoDeploy consumers.

Remove the stale waive entry now that the test passes end-to-end
(MMLU 84.75, GSM8K 88.97, MMMU 59.89 in 558s on 4xH100).

Signed-off-by: handongl <handongl@nvidia.com>
@trtllm-agent
trtllm-agent requested review from a team as code owners July 25, 2026 12:00
@coderabbitai

coderabbitai Bot commented Jul 25, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 7255872f-9321-4630-a51f-a8a87778f9ba

📥 Commits

Reviewing files that changed from the base of the PR and between cf44a1c and a90d01c.

📒 Files selected for processing (2)
  • examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml
  • tests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (1)
  • tests/integration/test_lists/waives.txt

Walkthrough

The Qwen 3.5 MoE auto-deploy configuration disables selected compile features and adds a compile-model setting. Its waived BF16 integration test is removed from the integration waiver list.

Changes

Qwen compile and test updates

Layer / File(s) Summary
Update Qwen compile settings
examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml		 Disables multi-stream MoE/GEMM compilation and piecewise compilation for the Qwen 3.5 MoE configuration.
Remove Qwen integration waiver
tests/integration/test_lists/waives.txt		 Removes the waiver for the Qwen 3.5 397B MoE BF16 small test.

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title is related to the main change, but it is truncated and could be more specific about the config and waived test cleanup.
Description check ✅ Passed The description explains the root cause, fix, and test plan, though it doesn't follow the template's exact section headings.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nvchenghaoz nvchenghaoz Awaiting requested review from nvchenghaoz nvchenghaoz is a code owner automatically assigned from NVIDIA/trt-llm-torch-autodeploy-devs
@QiJune QiJune Awaiting requested review from QiJune QiJune is a code owner automatically assigned from NVIDIA/trt-llm-devs
@schetlur-nv schetlur-nv Awaiting requested review from schetlur-nv schetlur-nv is a code owner automatically assigned from NVIDIA/trt-llm-devs
@StanleySun639 StanleySun639 Awaiting requested review from StanleySun639 StanleySun639 is a code owner automatically assigned from NVIDIA/trt-llm-qa
@EmmaQiaoCh EmmaQiaoCh Awaiting requested review from EmmaQiaoCh EmmaQiaoCh is a code owner automatically assigned from NVIDIA/trt-llm-infra-devs
@ZhanruiSunCh ZhanruiSunCh Awaiting requested review from ZhanruiSunCh ZhanruiSunCh is a code owner automatically assigned from NVIDIA/trt-llm-infra-devs

At least 1 approving review is required to merge this pull request.

Assignees

@suyoggupta suyoggupta

@trtllm-agent trtllm-agent

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@trtllm-agent @suyoggupta @HandongLi-01